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PREFACF. 


This  Note  report m on  work  conducted  at  Rand  on  "Air  Force  Health 
Delivery  Syatema."  The  research,  done  under  Project  AIR  FORCE,  rep- 
resents one  facet  of  an  analysis  of  the  role  that  physician  assistants 
wight  play  in  Air  Force  outpatient  clinics — the  quality  of  care  they 
deliver. 

This  Note  presents  an  analysis  of  data  collected  In  1974,  analysed 
In  1976,  and  reported  to  the  Air  Force  in  1977.  It  provides  background 
for  understanding  the  analysis  of  a more  extensive  data  collection  ef- 
fort carried  out  in  1977.  A more  recent  and  detailed  report,  dealing 
in  greater  depth  with  the  same  issue,  is  forthcoming  under  the  title 
"Physician's  Extenders  in  Air  Force  Primary  Medicine  Clinics:  Quality 
of  Care." 


KUMMAKY 


With  the  ending  of  the  physician  draft  and  the  subsequent 
decrease  in  physician  manning,  the  Air  Force  has  considered  alterna- 
tive staffing  patterns  in  which  physician  assistants,  working  under 
a doctor's  general  supervision,  take  care  of  some  patients  formerly 
treated  by  physicians.  This  Note  analyzes  the  quality  of 
care  rendered  by  these  physician  assistants,  using  as  a standard  of 
comparison  the  care  rendered  by  physicians  in  the  same  settings.  It 
focuses  on  the  technical  process  of  care  and  uses  data  collected  with 
a patient  contact  record  at  nine  Air  Force  bases  in  1974. 

Looking  at  a set  of  conditions  that  account  for  20  to  25  percent 
of  outpatient  visits  to  clinics  dealing  with  general  medicine  prob- 
lems, we  find  that  physician  assistants  are  performing  at  least 
as  well  as  physicians.  We  conclude  that  the  Air  Force  can  deliver 
the  same  quality  of  medical  care  when  physician  assistants,  working 
under  a doctor's  general  supervision,  treat  some  of  the  patients 
formerly  seen  by  physicians.  The  performance  of  these  physician 
assistants  constitutes  a strong  endorsement  of  the  Air  Force's  in- 
house  physician  assistant  training  program.  The  proven  quality  of 
this  program  should  be  weighed  in  any  decision  concerning  future 
sources  of  Air  Force  physician  assistants. 
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I . INTRODUCTION 


The  end  of  che  physician  draft  has  meant  a substantial  decrease 
in  the  number  of  physicians  available  to  the  Air  Force  for  the  pro- 
vision of  inpatient  and  ambulatory  (outpatient)  care.  The  Air  Force 
has  therefore  considered  alternative  staffing  patterns  which  permit 
the  available  physician  force  to  continue  providing  needed  medical 
care.  These  include  staffing  arrangements  in  which  highly  trained 
physician  extenders  (like  the  physician  assistant*)  work  in  combina- 
tion with  physicians. 

Medical  care,  if  delivered  successfully,  should  fulfill  four 
primary  constraints:  1)  it  should  be  cost-efficient  (the  economic 
constraint);  2)  it  should  be  administratively  feasible  (the  organiza- 
tional constraint);  3)  it  should  achieve  acceptance  by  patients  using 
the  system  (the  social  constraint);  and  4)  it  should  provide  care  of 
acceptable  quality  (the  medical  constraint).  This  Note 
addresses  how  the  fourth  constraint  is  affected  by  the  introduction 
of  highly  trained  physician  extenders  like  the  physician  assistant. 

It  considers  the  policy  issue:  Can  the  Air  Force  deliver  the  same 
quality  of  medical  care  when  physician  assistants,  working  under  the 
general  supervision  of  a physician,  treat  some  of  the  outpatients 
formerly  treated  by  physicians?  In  other  words,  can  a physician 
assistant  provide  a portion  of  the  care  formerly  provided  by  physi- 
cians, without  a sacrifice  in  the  level  of  quality?  While  primary 
care  nurse  practitioners  are  also  due  consideration,  they  are  not 
included  in  this  analysis  because  data  were  not  available.  The 
emphasis  in  the  Note  is  on  outpatient  care  because  this  is 
the  area  for  which  physician  assistants  have  generally  been  trained. 

* 

The  Air  Force  physician  assistant  is  a former  corpsman  who  has 
graduated  from  a two-year  program  of  instruction  that  includes  one 
year  of  classroom  work  in  the  basic  sciences,  and  a one-year  rotation 
through  the  outpatient  departments  of  a large  Air  Force  hospital.  The 
physician  assistant  is  therefore  trained  to  diagnose  and  treat  common 
illnesses,  and  can  also  help  manage  complex  patient  problems  under  the 
supervision  of  a physician. 
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II.  TTHOD  AND  RESULTS 


In  approaching  the  research  question  of  the  quality  of  care 
rendered  by  physician  assistants,  we  used  a comparative  method.  We 
compared  the  physician  assistants'  quality  of  care  with  physicians' 
quality  of  care,  for  conditions  that  physician  assistants  (as  well  as 
physicians)  are  trained  to  treat.  The  comparative  approach  allows  us 
to  examine  how  well  the  physician  assistant  measures  up.  It  permits 
us  to  approach  the  question  of  whether  or  not  physician  assistants' 
care  is  at  least  as  good  as  care  provided  by  physicians,  at  least  for 
the  type  of  routine  outpatient  problems  that  physician  assistants  are 
trained  to  treat. 

Specifically,  we  compared  physician  assistants  and  physicians  in 
two  ways:  1)  the  provision  of  appropriate  quality  of  care,  as  measured 
in  terms  of  "technical  process"  of  care,  and  2)  the  degree  of  misorder- 
ing  or  overutilization  of  tests  or  procedures,  in  which  a pattern  of 
ordering,  suggestive  of  wasteful  or  unnecessary  testing,  was  sought. 

The  data  source  for  this  Note  was  Rand's  1974  survey  of  outpatient 
care.  The  survey  used  the  Patient  Contact  Record  (PCR),  an  "encounter 
form".  (See  Appendix  A for  a copy  of  the  PCR.)  The  PCR  is  a check-off 
l is t filled  out  partly  by  the  patient  (or  parent),  and  partly  by  the 
provider(s)  of  care.  At  the  time  of  the  visit,  the  patient  filled 
out  requested  demographic  information,  while  the  provider  of  care 
checked  off  diagnostic  and  therapeutic  information. 

The  data  presented  come  from  nine  Air  Force  base  hospital  clinics, 
which  were  surveyed  by  use  of  the  PCR.  At  two  of  the  bases,  a PCR  was 
filled  out  for  each  patient  during  a six-month  period,  while  at  the 
remaining  seven  bases,  a PCR  was  filled  out  for  each  patient  during  a 
two-week  period.  The  descriptive  data  thus  gathered  came  from  all  of 
the  outpatient  clinics  at  each  base,  rather  than  from  selected  clinics. 
Our  analysis  pools  the  data  collected  at  the  nine  bases.  This  was 
possible  because  our  analysis  of  results  from  each  base  showed  no 
significant  variation  from  base  to  base,  for  the  questions  addressed 
In  this  Note.  It  is  reasonable  to  pool  the  data  from  all  bases,  so 
long  as  there  is  no  major  variation  from  base  to  base. 
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THE  DATA  COLLECTION  INSTRUMENT 

The  Patient  Contact  Record  is  a data  collection  instrument  known 
generically  as  an  "encounter  form".  Like  any  other  data  collection 
instrument,  the  encounter  form  has  both  strengths  and  limitations. 

The  greatest  advantage  of  the  encoirter  form  is  that  it  is  completed 
at  the  actual  moment  of  provision  of  care;  furthermore,  the  crucial 
information  is  furnished  by  the  actual  provider  of  care.  In  view  of 
the  fact  that  the  PCR  was  filled  out  for  a limited  time  period,  we 
are  reasonably  certain  that  the  providers  of  care  were  highly  moti- 
vated to  record  accurately  the  actual  care  delivered. 

There  are  also  limitations  to  an  encounter  form.  One  of  them 
is  that  the  provider  is  limited  to  those  diagnostic,  procedural  and 
therapeutic  categories  which  are  available  for  check  off.  In  addi- 
tion, it  is  possible  to  make  recording  errors  or  to  omit  an  item. 
Although  one  can  always  point  to  the  unverifiable  accuracy  of  diagno- 
sis, that  particular  limitation  pertains  to  virtually  all  methods 
currently  in  use  for  gathering  data  relevant  to  the  quality  of  medical 
care,  and  is  not  a special  liability  of  the  encounter  form.  Finally, 
the  1974  Rand  PCR  had  not  been  designed  with  the  specific  intention 
of  measuring  quality  of  medical  care. 

When  all  the  strengths  and  limitations  are  considered,  the  PCR 
was  felt  to  be  a very  good  means  of  gathering  data  from  outpatient 
areas.  The  strength  of  having  the  PCR  completed  by  the  provider  at 
the  actual  time  of  visit  cannot,  in  our  opinion,  be  overestimated. 

In  addition,  it  is  possible  that  some  of  the  limitations  of  the  PCR 
were  really  a blessing  in  disguise.  Because  of  the  limited  selection 
of  categories  available  for  check  off,  it  was  mandatory  on  our  part 
to  concentrate  on  straightforward,  stark,  simple  criteria  for  measur- 
ing  the  technical  process  of  care.  Thus,  as  will  be  seen,  there  is 
less  room  for  cavil  over  the  selected  criteria. 

The  PCR  also  compelled  us  to  concentrate  on  measuring  the  tech- 
nical process  of  care.  Quality  of  medical  care  has  traditionally 
been  organized  into  three  areas:  structure  (available  facilities, 
equipment,  and  personnel),  process  (what  is  done  to  or  for  the 
patient),  and  outcome  (what  happens  to  that  patient).  Quality  of 


t 
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medical  care  may  also  be  divided  into  "technical"  quality  and  "art  of 
care"  quality.  "Technical  process"  quality  is  relatively  easy  to 
measure,  but  is  always  open  to  the  question  of  the  connection  of  the 
process  of  care  with  the  eventual  outcome  of  care.  By  concentrating 
on  ordinary  conditions  of  low  or  moderate  complexity  (which  comprise 
the  majority  of  '«es  seen  in  Air  Force  outpatient  clinics),  we 
believe  we  avoided  the  potential  difficulties  in  concentrating  on 
measuring  "technical  process"  quality  of  care.  The  diagnostic  and 
therapeutic  actions  which  comprised  our  criteria  for  various  condi- 
tions quite  obviously  bear  a relationship  to  the  outcome  of  care. 

FORMAT  OF  RESULTS 

Table  1 is  labeled  First  Visit  Technical  Quality.  First,  let  us 
review  its  format.  The  far  left  column  lists  a number  of  conditions 
which  were  seen  in  outpatient  clinics.  The  diagnoses  are  common, 
everyday  problems — they  are  routine  outpatient  fare.  It  should  be 
noted  that  the  conditions  discussed  in  this  Note,  although 
few  in  number,  represent  between  20  and  25  percent  of  all  visits  made 
to  general  medicine  outpatient  areas.  The  simplicity  of  diagnoses 
also  guards  against  the  potential  problem  of  misdiagnosis,  a diffi- 
culty which  would  clearly  invalidate  the  application  of  quality  cri- 
teria. These  diagnoses  are  ordinary  enough  so  that  room  for  mis- 
diagnosis is  much  reduced  from  what  it  would  be  if  complex  outpatient 
or  inpatient  diagnoses  were  being  considered. 

The  second  column,  labeled  Good  Care  Criterion,  lists  the  cri- 
teria which  were  applied  to  the  corresponding  conditions  in  the  first 
column.  With  the  exception  of  warts,  all  the  criteria  listed  in  this 
table  concerned  the  first  visit  for  the  designated  condition.  It  is 
important  to  stress  that  we  present  all  of  the  criteria  that  we  could 
define,  for  which  there  was  a sufficient  number  of  observations  avail- 
able when  data  were  pooled  from  the  nine  bases.  Note  that  the  first 
four  criteria  represent  therapeutic  procedures — that  is,  a therapy 
that  was  employed  for  the  condition.  The  remaining  seven  criteria 
deal  with  diagnostic  procedures — that  is,  certain  procedures  that 
should  have  been  carried  out  because  they  are  needed  to  make  a 
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diagnosis  or  to  reach  a daclalon  regarding  management  of  the  patlant'a 
condlt Ion. 

The  next  two  columna,  labeled  MD  Compliance  and  PA  Compliance, 
praaent  the  data  for  the  number  of  tlmea  the  phyalclan  (HD)  and  the 
phyaldan  aaalatant  (PA)  carried  out  the  good  care  criterion  apeclfled. 
The  MD  Compliance  column  presents  caaea  seen  by  phyalclana  only,  and 
the  PA  Compliance  column  presents  patients  seen  by  phyalclan  assis- 
tants only — that  Is,  patients  seen  together  by  a physician  and  phy- 
sician assistant  are  excluded,  and  patients  seen  neither  by  a physi- 
cian nor  physician  assistant  are  also  excluded.  Finally,  the  right- 
hand  column  reports  the  presence  or  absence  of  statistical  signifi- 
cance at  the  P - 0.05  level. 


Table  1 


FIRST  VISIT  TECHNICAL  QUALITY 


Condition 

Impet lgo 

Otitis  media 

Acute  sinusitis 

Urinary  tract 
infection 
Pharyngitis 
(only) 
Pneumonia 

Urinary  tract 
infection 
Urinalysis  group 

Diabetes 

No  diagnosis  at 
this  time 
Warts 


Good  Care 
Criterion8 

Prescrip- 

tion 

Prescrip- 

tion 

Prescrip- 

tion 

Prescrip- 
tion 
Throat 
culture 
Chest 
x-ray 
U/A  or 
culture 
Urinalysis 

Blood 

sugar 

CBCb 

No  more 
than  2 
visits 


MD  Compliance 

PA  Compliance 

Statistical 

Significance? 

86* 

n*51 

l00ln-26 

Yes— PA 

80*  , ™ 
n»430 

9Un-180 

Yes— PA 

78t  m 
n“113 

93*  .. 

n“55 

Yes— PA 

“Vm 

77%  __ 

n«77 

N.S. 

70* 

n-155 

Yes— PA 

521  ... 
n*56 

67% 

n*12 

N.S. 

61*  , n . 
n-191 

79*  .. 

n«77 

Yes— PA 

41* 

n“400 

521  ,AA 

n“164 

Yes— PA 

42* 

n-43 

29* 

n-7 

N.S. 

21%  . c 7 

n*457 

23* 

n-270 

N.S. 

86%  UO 

n»142  pts • 

85* 

n«53  pts. 

N.S. 

aAU  criteria  (except  warts)  concern  the  first  visit  for  this 
condition. 


b 


Complete  blood  count. 
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TECHH1CAL  QUALITY  OF  CARE:  RESULTS 

Glancing  first  at  the  statistical  significance  column,  we  note 
a clear  pattern.  Physician  assistants  are  performing  at  least  as 
well  as  physicians,  at  least  when  quality  is  measured  by  the  criteria 
we  have  available,  for  the  kinds  of  routine,  outpatient  problems  which 
they  are  trained  to  treat. 

Let  us  examine  the  conditions  one  by  one.  The  first  criterion 
states:  if  the  patient  had  impetigo,  then  a prescription  should  have 
been  written  on  the  first  visit  (unfortunately,  the  PCR  was  not  spe- 
cific enough  for  us  to  know  what  kind  of  prescription  was  written; 
therefore,  we  have  to  be  satisfied  with  information  on  whether  or  not 
there  was  at  least  some  prescription  written).  For  86  percent  of 
impetigo  patients  seen  by  physicians,  the  physician  wrote  a prescrip- 
tion, and  for  100  percent  of  impetigo  patients  seen  by  physician 
assistants,  the  physician  assistants  wrote  a prescription.  The  diff- 
erence is  statistically  significant. 

For  otitis  media  (Inflammation  of  the  middle  ear,  a common  prob- 
lem in  children),  a prescription  should  have  been  written  during  the 
first  visit  when  the  condition  was  checked  off  by  the  provider.  Look- 
ing at  large  numbers  of  patients  (430  seen  by  physicians  and  180  seen 
by  physician  assistants),  we  note  that,  in  most  cases,  a prescription 
was  indeed  written  by  physicians  and  by  physician  assistants.  How- 
ever, physician  assistants  wrote  a prescription  significantly  more 
often  than  physicians.  What  should  we  make  of  the  fact  that  neither 
the  80  percent  nor  91  percent  is  100  percent?  Shouldn't  100  percent 
of  patients  with  otitis  media  have  a prescription  written  on  the  first 
visit?  It  would  likely  be  the  consensus  of  a group  of  experts  that 
this  is  so.  However,  there  are  good  reasons  why  we  should  not  expect 
a 100  percent  performance.  For  one  thing,  it  is  perfectly  possible 
that  the  provider  occasionally  forgot  to  check  a box,  or  checked  the 
wrong  box  on  the  PCR.  Secondly,  it  is  perfectly  possiible  that  the 
provider  was  about  to  write  a prescription  when  "he  parent  said,  "Oh, 
don't  bother  to  write  a prescription,  we  have  s*  ne  of  that  medicine 
left  at  home  from  the  last  time."  For  these  and  other  reasons,  it 
is  always  unrealistic  to  expect  100  percent  compliance  with  any 
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criterion.  Both  the  80  percent  end  91  percent  levela  are  very  high, 
and  auggeat  that  preacriptiona  were  being  written  for  otitia  media  in 
virtually  all  casea  by  both  groupa  of  provldera.  Note  that  the  PCR 
grouped  aeroua  otitia  media  and  purulent  otitia  media  into  one  cate- 
gory. Although  the  type  of  medicine  preacrlbed  may  aometimea  be 
different  for  theae  two  conditiona,  both  conditions  do  require  the 
preacribing  of  a medicine,  ao  our  criterion  la  a valid  one. 

If  a patient  haa  acute  alnualtia  and  la  being  aeen  for  the  flrat 
time  for  thla  condition,  a preacrlption  ahould  be  written.  ThlB  was 
done  very  often  by  both  groups,  and  significantly  more  often  by  phy- 
sician assistants. 

It  la  generally  accepted  that  if  the  diagnosis  of  urinary  tract 
infection  la  made,  some  medicine  should  be  prescribed  at  the  time  of 
initial  presentation  with  the  infection.  In  fact,  a preacrlption  was 
written  for  urinary  tract  Infection  quite  often  by  both  groupa,  with 
no  statistically  significant  difference. 

Now  we  turn  to  dlagnoatlc  procedures.  If  the  provider  is  to 
make  a diagnosis  of  pharyngitis  (sore  throat),  and  aore  throat  is  the 
only  problem  checked  off  by  the  provider  (that  is,  runny  nose,  fever, 
or  cough  was  not  checked  in  addition),  then  a throat  culture  ahould 
be  obtained  to  make  certain  that  the  throat  has  not  been  colonized 
with  streptococcus,  a bacterium  for  which  treatment  with  an  anti- 
biotic la  essential.  Both  physicians  and  physician  asalatanta  saw 
large  numbers  of  patients  with  pharyngitis  alone  (the  fact  that  the 
number  of  patients  was  identical  for  the  two  groups  is  coincidental). 
Physician  assistants  outperformed  physicians  in  obtaining  throat  cul- 
tures. Again,  we  might  ask:  Are  the  52  percent  and  70  percent  fig- 
ures acceptable  in  absolute  terms?  We  are  fortunate  in  having  com- 
parable data  from  the  civilian  sector  for  this  criterion.  In  1976, 
Kane  and  coworkers  fielded  an  encounter  form  among  physician  precep- 
tors and  their  MEDEX  (a  type  of  physician  aasistant)  in  private 
offices  throughout  the  state  of  Utah.  The  data  from  Utah  showed  that 
Utah  physicians  obtained  throat  cultures  for  pharyngitis  58  percent 
of  the  time  (compared  to  the  Air  Force  physicians'  rate  of  52  percent) 
while  the  MEDEX  in  Utah  obtained  throat  cultures  for  pharyngitis 
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68  percent  of  the  tine  (compered  to  the  Air  Force  phyelclen  assls- 

* 

tents  rate  of  70  percent).  These  percentages  show  that  Air  Force 
performance  levels  are  comparable  to  levels  reached  in  the  civilian 
sector.  This  similarity  of  levels  also  bolsters  our  confidence  in 
the  rest  of  our  observations. 

If  a patient  is  seen  in  the  outpatient  department  and  the  diag- 
nosis of  pneumonia  is  made  for  the  first  time,  a chest  x-ray  should 
be  performed.  This  criterion  would  seem  to  be  an  absolute  one,  but 
that  is  not  necessarily  so.  For  patients  whose  diagnosis  of  pneusronia 
is  being  made  for  the  first  time  in  the  outpatient  area,  it  is  diffi- 
cult to  imagine  how  the  diagnosis  could  be  properly  made  without 
obtaining  a chest  x-ray.  However,  it  is  possible  that  certain  patients 
seen  for  the  first  time  in  the  outpatient  area  with  pneumonia  were 
actually  being  followed  up  after  a recent  hospitalization  for  pneu- 
monia. In  the  case  of  a follow-up  visit,  the  provider  of  care  might 
have  decided  that  a chest  x-ray  was  not  yet  needed,  because  the  patient 
had  been  in  the  hospital  so  recently.  Thus,  we  would  not  expect  high 
percentage  compliance  with  this  criterion.  In  fact,  physicians  and 
physician  assistants  performed  at  approximately  the  same  level  in 
obtaining  chest  x-rays  for  patients  with  pneumonia  (the  number  of 
patients  seen  by  physician  assistants  is  quite  small,  however). 

It  is  generally  agreed  that,  if  the  diagnosis  of  urinary  tract 
infection  is  to  be  made,  some  objective  evidence  of  the  urinary  tract 
infection  must  be  obtained.  That  is,  it  is  not  acceptable  to  make 
the  diagnosis  and  begin  treatment  on  the  basis  of  symptoms  alone. 
Therefore,  we  specified  that  either  a urinalysis  or  urine  culture  had 
to  be  obtained  on  the  first  visit  at  which  the  diagnosis  of  urinary 
tract  infection  was  made.  Note  that  many  experts  would  require  both 
tests,  or  at  least  s urine  culture,  in  order  to  make  the  diagnosis; 
however,  we  agreed  to  accept  either  a urinalysis  or  a culture.  In 
fact,  61  percent  of  patients  seen  by  the  physician  had  a urinalysis 
or  a culture  when  the  diagnosis  of  urinary  tract  infection  was  being 


Vane,  R.  L.,  Olsen,  D.  M.,  Castle,  C.  H.:  Medex  and  their 
physician  preceptors.  J Are  Med  Assoc  236:2509-2512,  1976 
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made,  as  compared  to  79  percent  of  thoae  patients  seen  by  phyalclan 
assistants.  The  difference  is  statistically  significant. 

The  "urinalysis  group"  refers  to  a group  of  diagnoses  for  which 
it  Is  likely  that  obtaining  a urinalysis  would  contribute  to  good 
care.  The  urinalysis  group  Includes  not  only  urinary  tract  infec- 
tion, but  also  diagnoses  such  as  urolithiasis  (stones),  urethritis, 
benign  prostatic  hypertrophy,  and  other  diseases  of  the  urinary  sys- 
tem. Because  it  Is  more  broadly  defined,  this  criterion  is  also  more 
questionable.  The  degree  of  compliance  would  be  expected  to  be  lower, 
as  in  fact  it  is. 

It  was  reasoned  that  if  the  diagnosis  was  diabetes  mellltua,  a 
blood  sugar  test  should  have  been  obtained  on  the  first  visit  for 
the  condition.  It  was  not  done  very  often,  which  might  be  explained 
by  the  fact  that  the  patient  had  been  transferred  from  some  other 
facility,  or  was  being  followed  up  after  hospitalisation.  In  any 
case,  there  is  no  statistically  significant  difference,  although  the 
number  of  patients  with  this  condition  seen  by  physician  assistants 
is  very  small. 

"No  diagnosis  at  this  time"  was  a bona  fide  check-off  category 
on  the  Patient  Contact  Record.  It  was  there  to  permit  providers  who 
were  still  quite  uncertain  of  the  diagnosis  not  to  have  to  check  a 
more  specific  diagnosis.  Me  reasoned  that  if  the  patient  presented 
such  a riddle  that  no  diagnosis  could  be  made  at  the  time  of  the  first 
visit,  then  a complete  blood  count  (which  includes  hematocrit  or 
hemoglobin,  white  blood  count  and  differential  cell  count)  would  be 
warranted.  The  data  show  that  for  large  numbers  of  patients,  physi- 
cians and  physician  assistants  obtained  the  CBC  at  approximately  the 
same  rate. 

Finally,  we  set  up  a different  kind  of  good  care  criterion: 
treatment  of  warts  should  require  no  more  than  two  visits.  In  fact, 
virtually  all  cases  of  warts  were  handled  in  one  or  two  visits,  and 
were  handled  with  equal  dispatch  by  physicians  and  physician  assis- 
tants. 

Summarizing  the  information  in  the  first  table,  the  pattern  of 
care  shows  that  physician  assistants  are  performing  at  least  as  well 


as  physicians,  insofar  as  we  can  measure,  when  they  treat  these  common 
conditions. 

Host  conditions  in  outpatient  medicine  are  acute,  so  that  atten- 
tion to  the  first  visit  alone  makes  sense.  However,  there  are  certain 
conditions,  whether  acute  or  chronic,  for  which  it  is  equally  reason- 
able or  more  reasonable  to  look  at  an  entire  episode  of  care,  in 
which  case  a criterion  would  require  the  provider  to  perform  a certain 
action  on  at  least  one  of  a number  of  visits.  Table  2 addresses  the 
issue  of  technical  quality  of  care  rendered  over  time.  The  data  come 
only  from  the  two  bases  where  data  were  collected  for  a complete  six- 
month,  rather  than  a two-week,  period.  The  question  here  is:  was 
the  needed  test  ordered  if  it  had  not  been  ordered  before?  What  we 
have  before  us  is  a "batting  average"  for  each  type  of  provider:  the 
number  of  times  the  correct  thing  was  done  divided  by  the  number  of 
opportunities  to  do  the  correct  thing.  The  formal  fraction  can  be 
stated  as  follows: 

Number  of  times  the  provider  ordered  a needed  test 

Number  of  occasions  the  provider  should  have  ordered  the  test. 

In  other  words,  of  the  times  a patient  appeared  in  the  office  not  yet 
having  received  a needed  test,  how  often  did  the  physician  or  physi- 
cian assistant  order  that  needed  test? 

It  is  important  to  consider  that,  in  applying  the  criteria  listed 
in  Table  2,  we  are  viewing  episodes  of  illness  exclusively  from  the 
beginning.  That  is,  we  have  included  the  series  of  visits  made  by  a 
patient  only  if  we  see  the  very  first  as  well  as  subsequent  visits 
made  by  the  patient  for  a particular  condition. 

When  we  look  at  pneumonia,  we  find  that  in  46  percent  of  50 
visits  to  the  physician  where  an  x-ray  still  remained  to  be  performed, 
the  x-ray  was  ordered  by  the  physician.  Similarly,  in  40  percent  of 
the  ten  visits  when  a patient  came  to  the  physician  assistant  still 
needing  an  x-ray  for  pneumonia,  that  x-ray  was  ordered  by  the  physi- 
cian assistant.  The  difference  is  not  statistically  significant. 
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Tabic  2 

i 

TECHNICAL  QUALITY  OF  CARE  OVER  TIME:  HAS  A NEEDED 
TEST  ORDERED  IF  NOT  ORDERED  BEFORE? 


Compliance 

Statistical 

Condition  Good  Cara  Criterion  MD  PA  Significance? 


Pneumonia 

Did  x-ray.  If  not  done 
since  Illness  began 

46Z 

(n  ■ 50) 

40Z 
(n  - 

10) 

N.S. 

UTI 

Did  U/A  or  culture, 
if  not  done  since 
illness  began 

47Z 

(n  - 173) 

81Z 
(n  - 

69) 

Yes— PA 

Urinalysis 

group 

Did  U/A,  if  not  done 
since  illness  began 

3QZ 

(n  - 396) 

50Z 
(n  - 

148) 

Yes— PA 

Diabetes 

Ordered  sugar,  if 
not  ordered  before 

47Z 

(n  - 287) 

31Z 
(n  - 

26) 

N.S. 

Anemia 

Ordered  CBC,  If 
not  ordered  before 

60Z 

(n  - 78) 

70Z 
(n  ■ 

10) 

N.S. 

12- 


For  urinary  tract  infection  and  the  ordering  of  a urinalysis  or 
culture,  we  again  see  the  consistently  superior  perfornance  of  physi- 
cian assistants  in  the  area  of  ordinary  urinary  tract  disease.  Simi- 
larly, for  the  urinalysis  group,  physician  assistants  were  signifi- 
cantly more  likely  to  order  a urinalysis  which  needed  to  be  ordered. 

If  we  look  at  the  number  of  diabetics  for  whom  we  can  observe 
Initial  and  subsequent  visits,  we  note  that  physicians  still  order 
necessary  blood  sugars  more  often  than  physician  assistants,  although 
the  difference  is  not  statistically  significant. 

Finally,  both  groups  ordered  CBCs  fairly  often  when  they  were 
still  needed  in  patients  who  carried  the  diagnosis  of  anemia. 

To  summarize  Tables  1 and  2,  it  appears  that  physician  assistants 
are  rendering  an  acceptable  quality  of  technical  medical  care.  We 
will  return  to  this  point  after  we  examine  the  next  question,  that  of 
"mis-ordering". 

UTILIZATION  OF  TESTS  AND  PROCEDURES:  RESULTS 

Appropriate  ordering  of  tests  and  procedures  is  another  important 
facet  of  good  quality  medical  care.  This  section  examines  the 
question  of  appropriate  utilization  of  tests  and  procedures  in  two 
ways:  first,  by  comparing  utilization  rates  for  physicians  and 
physician  assistants  across  a number  of  tests  and  procedures;  and 
second,  by  selecting  specific  tests  or  procedures  (e.g.,  electro- 
cardiogram, physical  therapy)  and  analyzing  whether  or  not  they  were 
employed  in  situations  likely  to  have  been  inappropriate.  As  we 
shall  see,  the  first  method  is  highly  objective,  but  the  second 
method  reveals  more  about  the  appropriateness  of  utilization. 

We  begih  our  examination  of  appropriateness  of  utilization  by 
computing  the  number  of  times  a specific  test  or  procedure  was 
ordered  during  100  visits  for  a certain  diagnosis  or  group  of 
diagnoses.*  Although  it  might  be  unwise  to  make  value  judgments  con- 
cerning the  suitable  number  of  times  that  ordering  should  occur,  we 


Appendix  C lists  the  diagnoses  comprising  each  group. 


would  be  troubled  If  physicians  and  physician  assistants  were 
ordering  at  widely  varying  rates.  These  rates  are  also  helpful  to 
the  adninistrator  who  wants  to  predict  the  number  of  tests  that  he 
can  expect  the  staff  to  order. 

With  data  collected  during  our  1974  survey,  we  have  examined 
rates  of  ordering  for  the  following  tests  or  procedures:  complete 
blood  count  , urinalysis,  any  type  of  x-ray,  electrocardiogram,  serum 
chemistry,  and  physical  therapy.  Table  3 shows  gross  rates  of  order- 
ing these  tests  or  procedures  per  100  visits  for  patients  seen  only 
by  a physician,  or  only  by  a physician  assistant.  Tests  ordered  when 
a patient  saw  both  or  neither  of  these  two  types  of  providers  on  a 
visit  are  excluded.* 

Reviewing  the  rates,  it  is  immediately  apparent  that  there  are 
no  large  absolute  differences  in  rates  of  ordering  for  all  problems 
taken  as  a group.  For  example,  with  the  complete  blood  count,  we 
find  that  physician  assistants  are  ordering  one  more  complete  blood 
count  per  100  visits  than  are  physicians.  Even  though  this  differ- 
ence is  statistically  significant,  it  is  doubtful  that  there  is  any 
clinical  or  economic  significance  that  can  be  attached  to  one  ad- 
ditional complete  blood  count  per  100  visits.  We  note  that,  for 
three  of  these  tests,  physician  assistants  order  significantly  more 
tests,  while  physicians  order  significantly  more  often  for  one  of  the 
tests. 

However,  this  gross  calculation  does  not  get  at  the  variables 
that  may  influence  the  ordering  of  tests.  It  does  not  really  tell 
us  if  physician  assistants  are  overutilizing  or  underutilizing  tests 
and  procedures  in  comparison  to  physicians.  What  variables  may 
affect  the  rates  of  ordering  tests?  One  is  the  patient  mix:  one 
type  of  provider  might  over-order  a given  test  for  one  group  of 
patients  — yet,  because  it  sees  a smaller  number  of  patients  in  this 
group,  it  might  end  up  with  the  same  gross  ordering  rate  as  another 

* 

The  number  of  visits  varies  slightly  from  test  to  test  due  to 
minor  data  inconsistencies. 
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Table  3 


RATES  OF  ORDERING  TESTS  PER  100  VISITS 


Test 

Rate 

MD 

No.  of 
Visits 

Rate 

PA 

No.  of 

Visits 

i 1 

' 

Complete  blood  count 

5.70 

30349 

6.57^ 

7487 

2 

Urinalysis 

5.70 

30350 

7.41° 

7487 

• 

Any  x-ray 

9.00 

30345 

8.51 

7485 

• 

Electrocardiogram 

2.01a 

30350 

1.42  ^ 

7487 

Serum  Chemistry 

2.33 

30350 

2.93° 

7487 

i 

“1 

Physical  therapy 

0.86 

30350 

0.81 

7487 

ij 

y 

Significant  p £ .05. 
^Significant  p £ .01. 
Significant  p < .001. 
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type  of  provider  that  orders  appropriately  for  a larger  number  of 
patients  In  this  group.  Another  variable  is  the  proportion  of  Initial 
vlalts  to  return  visits  for  a particular  problem.  If  the  ratio  of 
Initial  visits  to  return  visits  differc  for  physicians  as  opposed  to 
physician's  assistants,  the  rate  of  ordering  tests  might  well  vary, 
because  it  is  likely  that  more  procedures  would  be  ordered  on  an 


Initial  visit. 

Only  after  these  two  variables  are  taken  into  account  can  we 
perform  a more  exacting  analysis  of  different  rates  of  ordering  tests. 
Even  then,  we  will  be  left  with  the  question  of  the  reason  for  any 
observed  differences  in  ordering  rates  among  different  groups  of 
providers.  If  higher  ordering  rates  exist  for  one  group  as  compared 
to  another,  will  that  be  waste,  or  reasonable  camtion,  or  simply 
higher  quality?  We  will  have  reached  that  ill-defined  middle  mone 
where  quality  and  utilization  are  Inseparable.  The  tables  that  fol- 
low specify  a narrower  range  of  diseases,  in  order  to  lessen  the 
effect  of  patient  mix  on  the  results.  Likewise,  they  separate  Initial 
from  return  visits. 

As  we  turn  to  disease-specific  rates  of  ordering  tests,  let  us 
start  with  x-rays.  Table  A controls  for  disease  category.  "X-ray 
conditions"  is  the  general  term  for  a group  of  disorders,  all  of 
which  might  well  require  x-rays  at  some  time,  for  some  proportion  of 
patients  being  seen.  The  four  specific  disease  conditions  listed, 
i.e.,  tension  headache,  acute  and  chronic  sinusitis,  backache,  and 
arthritis  and  joint  pain,  are  even  more  specific  subsets  of  the 
general  term  "x-ray  conditions".  This  table  limits  the  count  of 
visits  to  those  encounters  for  which  the  specific  diagnosis  was  the 
sole  one  listed  on  the  Patient  Contact  Record.  Now  that  we  control 
for  case  mix,  we  notice  dramatic  changes  in  the  numbers.  Although, 
in  Table  1,  physicians  showed  a slightly  higher  rate  of  ordering 
x-rays  for  all  diagnostic  categories,  we  note  that  physician 
assistants  have  a slightly  (but  not  significantly)  higher  rate  of 
ordering  x-rays  for  "x-ray  conditions".  In  fact,  they  order  more 
x-rays  for  each  of  the  diagnostic  subsets,  and  significantly  more 
x-rays  for  acute  and  chronic  sinusitis  and  arthritis  and  joint  pain. 


■ 

I 
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Table  4 


X-RAYS  PER  100  VISITS,  FIRST  AND  RETURN 


MD 

PA 

No.  of 

No.  of 

No.  of 

No.  of 

Condition 

X-rays 

Visits 

X-rays 

Visits 

"X-ray  conditions" 

20.65 

6236 

22.61 

1265 

Tension  headache 

8.16 

245 

10.45a 

67 

Sinusitis  (acute  and  chronic) 

12.02 

341 

19.69 

127 

Backache 

13.87 

757 

15.84 

284 

Arthritis  and  joint  pain 

22.11 

606 

33.79° 

219 

a Significant  p <_  .05. 

Significant  p <_  .01. 

^Signlf leant  p <_  .001. 

Table  5 

X-RAYS  PER  100  VISITS:  DEFINITE  FIRST 
RETURN  VISITS  ONLY 

AND 

MD 

PA 

No.  of  No.  of 

Condition  X-rays  Visits 

No.  of 
X-rays 

No.  of 
Visits 

First  Visits  Only 


"X-ray  conditions" 

23.5 

2055 

27. 9a 

612 

Tension  headache 

11.4 

105 

11 

36 

Sinusitis  (acute  and  chronic) 

6.8 

118 

18. 5a 

65 

Backache 

16.4 

272 

23.3  b 

133 

Arthritis  and  joint  pain 

28.9 

204 

44.  r 

120 

Return  Visits  Only 


"X-ray  conditions" 

16.2 

2958 

15.1 

476 

Tension  headache 

4.8 

105 

9.1 

22 

Sinusitis  (acute  and  chronic) 

14.4 

153 

22.2 

45 

Backache 

9.2 

325 

8.7 

115 

Arthritis  and  joint  pain 

13.8 

269 

16.7 

72 

aSignlf leant  p £ .05. 


^Significant  p < .01. 
°Slgnif leant  p < .001. 
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Is  this  overutilization,  or  is  it  higher  quality  of  care? 

Table  5 permits  us  to  get  closer  to  the  answer  (but  not  to  reach 
it)  by  controlling  for  definite  first  viilts  versus  definite  return 
visits.*  When  we  look  at  the  differences  in  ordering  rates  for  firat 
visits  and  return  visits,  it  is  apparent  that,  with  the  single  ex- 
ception of  acute  and  chronic  sinusitis,  x-ray  rates  are,  as  expected, 
higher  for  first  visits  than  for  return  visits.  In  the  case  of 
sinusitis,  the  ordering  rates  for  both  physicians  and  physician 
assistants  Increase  on  return  visits,  which  suggests  that  both  types 
of  providers  become  more  concerned  if  the  patient  is  not  progressing 
satisfactorily  at  the  time  of  a return  visit.  For  both  first  visits 
and  return  visits,  physician  assistants  generally  order  more  x-rays 
for  these  conditions  than  do  physicians,  although  the  differences 
narrow  (with  the  exception  of  tension  headache)  upon  return  visits. 

For  sinusitis,  arthritis  and  joint  pain,  there  are  large  and 
significant  differences  in  x-ray  rates  on  first  visits.  In  the  case 
of  sinusitis,  it  is  impossible  to  guess  if  the  patients  required  the 
x-rays.  For  arthritis  and  joint  pain,  the  phyiician  assistants' 
ordering  rate  of  more  than  44  x-rays  per  100  visits  is  so  high  that 
one  might  well  question  whether  all  the  x-rays  were  necessary. 

Turning  now  to  a procedure,  physical  therapy  (Tables  6 and  7),  we 
note  that  physicians  order  more  physical  therapy  for  a group  of  con- 
ditions where  it  might  well  be  warranted.  The  difference  is  not 
statistically  significant.  The  two  sub-groups  selected  for  physical 
therapy — arthritis  and  joint  pain,  and  backache — also  show  higher 
ordering  rates  for  physicians.  These  differences  persist  when  first 
and  return  visits  are  looked  at  separately  (Table  7),  although  now 
physician  assistants  order  slightly  more  physical  therapy  for 
arthritis  and  Joint  pain  on  first  visit.  It  seems  clear  that  physician 
assistants  are  not  overloading  the  physical  therapy  department  in 
comparison  to  physicians.  On  the  other  hand,  it  is  quite  possible 


* 

In  Table  3,  the  total  number  of  visits  is  lower  than  in  previous 
x-ray  tables  because  we  have  discarded  all  visits  with  ambiguous  first 
or  return  status,  or  vith  unindicated  first  or  return  status.  We  in- 
clude only  those  visits  which  we  are  certain  are  first  visits  or  return 
visits. 
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Table  6 


PHYSICAL  THERAPY  PER  100  FIRST 
AND  RETURN  VISITS 


MD 

PA 

No.  of 

No.  of 

Condition 

Rate 

Visits 

Rate 

Visits 

"Physical  therapy  group" 

5.26 

4281 

4.82 

871 

Arthritis  and  joint  pain 

4.11 

608 

1.83 

219 

Backache 

6.89 

755 

6.67 

285 

Table  7 


PHYSICAL  THERAPY  PER  100  VISITS:  FIRST  VISITS 
VERSUS  RETURN  VISITS 


MD 

PA 

No.  of 

No.  of 

Condition 

Rate 

Visits 

Rate 

Visits 

First  Visits  Only 

"Physical  therapy  group" 

5.6 

1475 

4.6 

483 

Arthritis  and  joint  pain 

2.4 

205 

3.3 

120 

Backache 

8.5 

272 

5.3 

126 

Return  \ 

Visits  Only 

"Physical  therapy  group" 

5.8 

1942 

5.4 

315 

Arthritis  and  joint  pain 

4.1 

270 

0.0 

72 

Backache 

4.6 

324 

8.6 

106 

a 
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that  physician  assistants  are  not  ordering  ae  much  physical  therapy 
as  might  be  consistent  with  good  quality.  If  there  were  reason  to 
suspect  that  this  were  the  case,  it  would  be  possible  to  review  a 
group  of  charts  bearing  specific  diagnoses  to  decide  if  a problem 
existed. 

Table  8 begins  more  detailed  consideration  of  electrocardiogram 
per  100  visits.  Unlike  the  gross  ordering  rate  for  electrocardiograms, 
in  which  physicians  ordered  more,  when  we  look  at  the  "BCG  conditions", 
we  now  see  physician  assistants  ordering  electrocardiograms  at  a 
slightly  higher,  but  not  statistically  significant  rate.  The  subset 
entitled  ischemic  heart  disease  and  other  heart  disease  shows  an  even 
greater  difference  in  the  rate  of  ordering  electrocardiograms,  but 
the  number  of  visits  handled  by  physician  assistants  la  too  small  to 
draw  any  conclusions  with  confidence.  Once  again,  we  note  that  or- 
dering rates  decrease  considerably  from  first  visit  to  return  visit 
both  for  physicians  and  for  physician  assistants.  The  most  reasonable 
conclusion  to  draw  Is  that  electrocardiograms  are  not  being  grossly 
ovarordered  by  physician  assistants,  and  that  there  would  be  little 
to  gain  by  investigating  further  In  this  area. 

Table  9 shows  ordering  rates  for  serum  chesilstries  or  electrolytes 
per  100  visits  when  the  analysis  is  limited  to  disease  conditions 
belonging  to  Che  "ECG  group".  Physician  assistants  consistently 
order  more  tests.  Whether  physicians  order  too  few,  or  physician 
assistants  too  many,  or  in  fact  if  there  is  a problem  at  all,  is  a 
question  which  might  be  selected  for  lnveatlgation. 

Finally,  Table  10  shows  that  the  difference  in  rate  of  ordering 
urinalysis  per  100  visits  for  all  conditions  is  intensified  when  the 
single  diagnosis  of  urinary  tract  infection  is  considered.  An  ln- 
veetigator  might  wish  to  examine  the  possible  reasons  for  this 
difference.  Such  investigation  would  require  chart  review. 

Review  of  utilisation  rates  for  ordering  tests  or  pvocedures  is 
disappointing,  in  that  definite  conclusions  cannot  be  reached.  Even 
after  case  mix  and  initial  versus  return  visit  are  controlled  for, 
it  still  remains  impossible  to  determine  if  we  are  dealing  with 
differences  attributable  to  different  quality  of  care  or  to  differing 
patterns  of  utilisation.  However,  utilisation  analysis  does  signal 
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Table  0 


ELECTROCARDIOGRAMS  PKR  100  VISITS 


MU 

PA 

No.  of 

No.  of 

Condition 

Rate 

Visits 

Kate 

Visits 

yiret  arid  He  turn  Viaita 

"KCG  group" 

Ischemic  and  other 

8.46 

2580 

10.28 

282 

24.65 

507 

37.5 

24 

FHrat  Haita  Only 

"KCG  group" 

Ischemic  and  other 

17.4 

431 

16.7 

54 

heart  disease 

37.8 

40 

33.3 

6 

Saturn  Via  it#  <.toly 

"KCG  group" 

Ischemic  and  other 

6.3 

1412 

7.5 

174 

heart  disease 

18.7 

331 

42. 4U 

14 

Significant  p £ .05. 


Table  9 


SERUM  CFEMISTRY— ELECTROLYTES  PER  100  VISITS 
FOR  MECG  GROUP"  ONLY 


Type  of  Visit 

MD 

PA 

Rate 

No.  of 
Visits 

Rate 

No.  of 
Visits 

First  and  return 

11.46 

2680 

20.0° 

280 

First  only 

13.7 

431 

23.1 

52 

Return  only 

8.4 

1912 

20.  la 

174 

Significant  p £ .05. 
^Significant  p £ .01. 
Significant  p £ .001. 


Table  10 

URINALYSIS  PER  100  VISITS  FOR  URINARY 
TRACT  INFECTION  ONLY 


Type  of  Visit 

MD 

PA 

Rate 

No.  of 
Visits 

Rate 

No.  of 
Visits 

First  and  return 

45.2 

598 

73.1fc 

160 

First  only 

59.6 

188 

78.9? 

76 

Return  only 

32.5 

277 

65. 3b 

72 

Significant  p £ .0. 
^Significant  p £ .001. 
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r 

potential  problem  area*.  If  desired,  it  would  be  possible  to  follow 
up,  by  means  of  chart  review,  any  significant  difference  discovered. 

Use  of  this  method  can  therefore  serve  as  a flagging  device  for 
uncovering  areas  where  chart  review  might  permit  more  certain  con- 
clusions to  be  drawn  concerning  the  presence  of  high  or  low  quality, 
or  too  high  or  too  low  utilisation  of  services. 

As  mentioned  at  the  beginning  of  this  utilisation  section,  there 
is  a second  method  for  investigating  possible  inappropriate  utilisation, 
i.e.,  selecting  specific  tests  or  procedures  and  analysing  whether  or 
not  they  were  employed  in  situations  likely  to  have  been  inappropriate. 
We  now  turn  to  this  aecond  method.  Table  11  presents  data  concerning 
patterns  of  possible  "mis-ordering".  The  word  "mis-ordering"  is  put 
in  quotation  marks  because  we  are  presenting  possible  rather  than 
definite  misutilisation  of  tests,  procedures,  or  therapy.  The  pro- 
cedures outlined  in  Table  11  were  probably  not  warranted,  given  the 
situation  specified.  However,  one  person's  ,(waste"  is  another  person's 
''reasonable  caution",  and  we  will  have  to  mention  this  caveat  again. 

The  first  two  procedures  presented  in  Table  11  form  a group. 

They  look  at  the  fraction: 

Number  of  procedures  ordered 

Number  of  patients  with  the  condition. 


Corysa  means  runny  nose,  and  it  is  unlikely,  when  runny  nose  was  the 
only  diagnosis,  that  a throat  culture  should  have  been  taken.  Al- 
though there  are  likely  to  be  some  exceptions,  we  would  like  to  see 
a very  low  rate  of  ordering  of  throat  cultures  in  patients  with 
runny  nose.  In  fact,  when  coryza  was  the  only  diagnosis,  a throat 
culture  was  infrequently  ordered  both  by  physicians  and  physician 
assistants,  both  groups  having  seen  a large  number  of  such  patients. 
The  difference  is  not  statistically  significant,  although  physician 
assistants  ordered  more  throat  cultures  than  physicians. 

Similarly,  it  is  unlikely  that  the  first  visit  for  asthma  is 
sufficient  in  and  of  itself  to  stimulate  the  provider  to  order  an 
x-ray.  In  fact,  under  these  circumstances  x-rays  were  infrequently 
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Table  11 

PATTERN  OF  POSSIBLE  "MIS-ORDERING" 


MD 

PA 

"Mls-ordering" 

"Mls-ordering" 

No.  of 

No.  of 

Statistical 

Procedure 

Rate  Instances 

Rate  Instances 

Significance? 
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ordered  by  both  physician  assistants  and  physicians,  although  the 
number  of  patients  seen  by  the  physician  assistants  is  small. 

The  next  two  criteria  presented  In  Table  11  comprise  another 
group.  The  fraction  is: 

Number  of  procedures  of  a specified  type  "not  needed'' 

Number  of  procedures  of  a specified  type  performed. 

The  question  addressed  by  the  first  criterion  in  this  group  is:  Were 
electrocardiograms  ordered  for  Inappropriate  diagnoses?  In  order  to 
approach  this  question,  we  first  made  a list  of  diagnoses  for  which 
electrocardiograms  are  likely  to  be  appropriate.  We  then  counted 
the  number  of  electrocardiograms  which  were  ordered  for  diagnoses 
which  did  not  appear  on  the  "appropriate"  list.  Of  387  electro- 
cardiograms ordered  by  physicians,  16  percent  of  them  were  ordered 
for  diagnoses  for  which  it  would  be  unusual  to  need  an  electro- 
cardiogram. For  physician  assistants,  28  percent  of  the  53  electro- 
cardiograms they  ordered  were  for  diagnoses  for  which  electrocardio- 
grams were  unlikely  to  be  appropriate.  The  difference  Is  statistically 
significant  in  favor  of  the  physicians. 

The  next  criterion  shows  that  physical  therapy  was  rarely  ordered 
for  inappropriate  diagnoses,  although  it  was  more  often  ordered  by 
physician  assistants. 

The  final  criterion  in  Table  11  concerns  Inappropriate  ordering 
of  x-rays.  The  fraction  here  is: 

Number  of  x-rays  ordered  on  visits  for  diagnoses  not  on  the 

"x-ray  appropriate"  list  

Total  number  of  visits  for  diagnoses  not  on  the  "x-ray 
appropriate" • list 

Again,  we  made  a list  of  diagnoses  for  which  taking  an  x-ray  is  llkelv 
to  be  appropriate.  We  then  looked  at  the  number  of  visits  for  all 
other  (x-ray  "not  needed")  diagnoses,  and  computed  how  often  x-ray* 
were  ordered  during  viilts  for  these  inappropriate  diagnoses.  «»n 
thousands  of  visits,  both  physicians  and  physician  assistants  'arely 
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ordered  x-reye  for  Inappropriate  diagnose*,  although  phyalclans  did 
order  then  slightly  more  frequently  (Inappropriately)  than  did 
physician  assistants. 

In  suemary,  looking  at  the  pattern  of  utilisation,  we  note  that 
In  four  of  the  five  situations  listed,  physician  assistants  did 
"misorder"  more  often  than  physicians.  However,  in  only  one  of  the 
situations  was  statistical  significance  reached.  Ue  conclude  that 
there  is  a mild  tendency  for  physician  assistants  to  order  more.  The 
question  must  again  be  raised:  Is  this  fairly  mild  degree  of  mls- 
ordering  to  be  called  "waste"  or  to  be  called  "reasonable  caution" 
on  the  part  of  providers  who  are  less  fully  trained  and  therefore 

1 • 

might  need  more  back-up,  which  they  could  obtain  by  ordering  more 
tests?  Of  course,  this  question  cannot  be  answered  with  the  data 
at  hand. 


: 


, 

! 

: 
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III.  CONCLUSIONS 


We  therefore  conclude  that  the  Air  Force  can  deliver  the  same 
quality  of  medical  care  when  physician  assistants  treat  some  of  the 
patients  formerly  treated  by  physicians,  and  that  no  quality  bar 
exists  to  the  continued  training  and  employment  of  physician  assis- 
tants in  Air  Force  outpatient  clinics.  In  a sense,  our  data  provide 
the  medical  go-ahead  for  physician  assistants.  We  find  the  quality 
of  care  they  deliver  to  be  acceptable  when  they  are  providing  care 
for  the  types  of  problems  they  have  been  trained  to  treat. 

Insofar  as  we  can  measure  with  our  criteria,  for  routine,  out- 
patient conditions,  physician  assistants  are  safe,  and  they  deliver 
a technical  process  quality  of  care  at  least  equal  to  that  of  physi- 
cians. Our  finding  that  physician  assistants  treat  at  least  as  well 

as  physicians  for  the  problems  we  were  able  to  study  is  consistent 

* 

with  other  published  evidence. 

There  is  a mild  tendency  for  physician  assistants  to  over-order 
some  tests  and  procedures.  If  the  over-ordering  is  truly  misordering 
(again  raising  the  question  of  waste  versus  reasonable  caution),  then  there 
appears  to  be  only  a mild  amount  of  misordering  of  some — not  all — 
tests  and  procedures  by  the  physician  assistants  as  compared  to  the 
physicians. 

The  performance  of  physician  assistants,  as  measured  by  these 
1974  data,  constitutes  a strong  endorsement  of  the  Air  Force's  in- 
house  physician  assistant  training  program.  It  would  behoove  the 
Air  Force  to  consider  the  proven  quality  of  its  in-house  training 
before  attempting  to  substitute  one  or  more  untested  civilian  training 
programs. 

Beyond  this,  any  further  work  on  physician  assistant  quality  of 
care  depends  on  new  data,  which  will  soon  be  forthcoming.  With  the 
approval  and  encouragement  of  the  Air  Force  Surgeon  General,  four 
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Sac  Appendix  B. 
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Air  Force  bases  commenced  a d aeons t rat Ion  prograa  proposed  by  Rand 
on  the  basis  of  Information  gathered  between  1974  and  1976.  Although 
the  daatonstration  prograa  consists  of  a nuaber  of  different  eleaents, 
the  aaln  one  Is  the  use  of  a nuch  richer  ale  of  physician  assistants 
and  prlaary  care  nurse  practitioners  in  outpatient  clinics.  At  these 
four  deaonstration  bases,  the  provision  of  general  medical  aabulatory 
care  is  carried  out  by  teams  of  providers  consisting  of  two  or  three 
physician  extenders  (physician  assistants  or  prlaary  care  nurse  prac- 
titioners) working  with  one  physician. 

The  demonstration  program  began  in  the  autumn  of  1976.  In  the 
spring  and  suamer  of  1977,  a revised  Patient  Contact  Record  was 
fielded  at  each  of  these  bases.  The  revised  PCR  is  specifically  de- 
signed to  provide  increased  information  relevant  to  the  issue  of  the 
everyday  quality  of  care  provided  by  physician  extenders  and  physicians 
in  Air  Force  outpatient  clinics.  In  addition  to  the  cos^arlson  of 
physicians'  and  physician  assistants'  performance,  such  as  was  possible 
with  the  1974  data,  we  will  also  take  the  opportunity  to  evaluate  the 
performance  of  the  first  few  classes  of  physician  assistants  produced 
by  the  Air  Force  as  compared  with  the  performance  of  the  more  recent 
graduates  of  the  training  program.  Finally,  we  should  also  be  able 
to  provide  some  information  on  the  performance  of  primary  care  nurse 
practitioners. 
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PART  II  Diagnostic  Information 
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PREVIOUS  RESEARCH  INTO  PHYSICIAN  EXTMTOFns'  quai.tty  nr  ring 

Since  the  Inception  of  formal  physician  extender  training  pro- 
gram* , the  question  of  the  quality  of  care  level  rendered  by  physi- 
cian extenders  has  not  been  slighted.  However,  If  one  looks  speci- 
fically at  the  technical  process  of  care  rendered  by  physician 
extenders  of  various  types  to  their  patients,  the  data  are  quite 
limited.  In  fact,  only  one  study  of  the  technical  process  of  care 
in  which  encounter  forms  were  used  as  the  method  of  data  collection 
has  been  reported  (Kane,  Olsen,  Castle,  1976). 

In  their  sweeping  evaluation  of  physician  extender  (PE)  programs, 
Appel  and  Lowln  (1975)  reported  that  "the  evidence  on  quality  of  care 
is  extensive  and  universally  favorable  to  the  PE.  The  recent  find- 
ings can  be  summarized  as  follows: 

(1)  When  patients  are  seen  by  PEs  and  later  by  physicians,  the 
diagnosis  and  treatment  is  almost  always  in  substantial  agreement. 

(2)  Over  time,  health  among  those  receiving  care  from  PEs  is 
comparable  to  or  better  than  that  of  those  receiving  conventionally 
delivered  services. 

(3)  Physicians  associated  with  PEs  almost  always  report  that 
the  quality  of  care  provided  by  PEs  Is  either  better  or  at  least 
comparable. 

(4)  Studies  show  that  PE  adherence  to  established  protocol  is 
as  good  as  or  better  than  that  of  physicians. 

(5)  PEs  refer  patients  to  the  supervising  physicians  when 
necessary,  and  tend  to  err  in  the  direction  of  referring  too  many 
rather  than  missing  those  who  need  to  be  seen  by  the  MD. 

(6)  Public  acceptance  of  PE  services  is  almost  universally 
favorable,  and  many  feel  that  the  quality  is  superior. 

(7)  Patient  management  by  physicians  and  PEs  has  been  judged 
to  be  roughly  equal  by  third  party  physician  review. 
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Before  turning  to  findings  specifically  concerned  with  measure- 
ment of  outcome  or  process  of  care,  it  is  worth  noting  a study  in 
which  a rating  of  job  performance  for  physician  assistants  was 
obtained  from  a rating  scale  completed  by  their  supervising  physicians. 
"Seventy-six  percent  of  the  662  supervising  physicians  surveyed  were 
'greatly  satisfied'  with  their  physician's  assistants  and  another  19 
percent  were  'moderately  satisfied.'  Only  5 percent  expressed  lesser 
degrees  of  satisfaction.  Similarly,  77  percent  of  these  supervising 
physicians  stated  that  they  would  'definitely'  rehlre  the  same  physi- 
cian's assistant  if  they  had  it  to  do  over  again  and  another  16  per- 
cent 'probably'  would.  Only  7 percent  Indicated  that  they  would  not 
hire  the  same  physician's  assistant."  Of  interest  is  the  fact  that 
"graduating  from  a military-sponsored  physician's  assistant  program 
appears  to  have  a beneficial  effect  upon  performance."  The  author 
could  not  distinguish  between  the  possibilities  that  the  training 
program  provided  by  the  military  is  superior,  or  that  military  physi- 
cian assistants  are  rated  more  favorably  because  they  have  been  given 
greater  responsibility  and  thus  find  themselves  in  "more  challenging 
and  demanding  clinical  situations"  (Perry,  1977). 

Many  findings  covering  PE  quality  are  not  directed  toward  tech- 
nical process  of  care  provided.  For  example,  the  outcome  of  care 
delivered  by  physician  extenders  has  been  studied  in  a number  of 
settings.  In  1969,  Lewis  et  al.  compared  regular  clinics  staffed  by 
physicians  with  nurse  practitioner  clinics  which  treated  comparable 
patient  populations.  There  were  no  differences  in  terms  of  deaths 
or  severity  of  disease  between  the  two  patient  groups,  but  there  were 
some  statistically  significant  differences  in  outcomes:  patients  in 
the  nurse  clinics  reported  relatively  decreased  disability  as  well  as 
decreased  discomfort  as  reflected  by  frequency  of  complaints.  In 
addition,  patients  in  the  nurse  clinics  had  statistically  significantly 
fewer  broken  appointments,  while  patients  in  regular  clinics  were 
significantly  more  critical  of  clinic  care  and  reported  significantly 
higher  rates  of  use  of  "other"  care  resources.  Lewis  et  al.  pointed 
out  that  these  clinics  were  ministering  primarily  to  chronically  ill 
patients,  where  the  supporting-role  functions  of  nurse  practitioners 
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might  wall  be  highly  consistent  with  the  majority  of  needs  of  the 
patients.  Ssckett  et  al.  (1974),  in  a private  office  setting,  were 
able  to  show  similar  levels  of  physical,  emotional,  and  social  func- 
tioning in  two  cos^arable  groups  of  patients  who  had  received  either 
nurse  practitioner  or  conventional  (physician)  care  for  one  full  year. 
Levine  et  al.  (1976)  compared  the  outcomes  of  medical  care  for  patients 
seen  by  ten  physicians  and  twelve  health  associates  delivering  ambu- 
latory care  in  a prepaid  group  practice.  Focusing  on  more  acute 
problems,  they  examined  such  outcomes  as  frequency  and  intensity  of 
pain  or  discomfort,  level  of  anxiety,  and  degree  of  activity  limita- 
tion, as  well  as  patient  satisfaction.  Their  analysis  suggested  that 
the  health  associates  were  "providing  care,  within  their  areas  of 
responsibility,  of  comparable  quality  to  that  delivered  by  physicians." 
Regarding  process  of  cars,  Levine  et  al.  compared  the  general  types 
of  care  (e.g. , acute  care,  chronic  care,  well  care),  rendered  by 
physicians  and  health  associates,  but  they  did  not  then  present  any 
data  concerning  the  quality  of  the  technical  process  of  care  rendered. 

Turning  to  technical  process  of  care,  the  data  are  distinctly 
more  limited.  Komaroff  (1974),  and  more  recently  Sullivan  (1976), 
showed  that  PEs  adhere  to  established  protocols  which  are  designed 
to  define  appropriate,  high  quality  diagnosis  and  management  of 
problems.  Sullivan  reports  that  the  introduction  of  check-list  pro- 
tocols for  sore  throats  raised  (for  both  physicians  and  physician 
extenders)  questioning  for  presence  of  drug  allergies  from  7 percent 
before  protocol  use  to  97  percent  after  protocol  use;  raised  perfor- 
mance of  throat  culture  from  83  percent  to  90  percent;  and,  also 
appropriately,  lowered  the  giving  of  antibiotics  from  45  percent 
before  using  the  protocol  to  16  percent  after  the  protocol.  Fine 
and  Silver  (1973)  assessed  the  comparative  diagnostic  abilities  of 
child  health  associate  interns  and  practicing  pediatricians  as  both 
types  of  providers  evaluated  143  ambulatory  pediatric  patients. 
Diagnostic  agreements  by  the  two  groups  occurred  in  91.6  percent  of 
cases;  in  4.9  percent  (seven  patients)  the  child  health  associate 
intern  listed  diagnoses  not  recorded  by  the  pediatrician;  in  3.5 
percent  (five  patients),  differences  occurred  in  their  diagnostic 
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lnterpretatlon.  The  authors  concluded  that  the  accuracy  with  which 
child  health  associate  interna  can  identify  common  pediatric  problems 
is  comparable  to  that  of  pediatricians. 

In  the  realm  of  management,  LoGerfo  at  al.  (1976),  in  comparing 
the  quality  of  ambulatory  care  provided  by  a prepaid  group  organiza- 
tion with  that  provided  by  the  fee-for-aervice  sector.  Included  the 
new  health  practitioners  (nurse  practitioners  and  Medex)  employed  by 
the  prepaid  group  in  their  calculations  and  evaluation.  However, 
the  care  rendered  by  the  prepaid  group  was  provided  by  generalist  and 
specialist  physicians  as  well  as  by  new  health  practitioners,  and  the 
new  health  practitioners'  performance  was  not  separately  reported,  so 
it  is  not  possible  to  reach  any  conclusion  about  the  quality  of  tech- 
nical care  provided  by  the  new  health  practitioners  considered  inde- 
pendently. In  general,  however,  there  was  no  significant  difference 
between  care  provided  in  the  prepaid  group  and  that  provided  in  the 
fee-for-service  setting.  When  trends  did  exist,  they  consistently 
favored  the  prepaid  group  practice,  which,  as  stated,  did  include  a 
number  of  new  health  practitioners. 

A recent  study  compared  the  management  of  patients  with  acute 
respiratory  Illnesses  by  Medex  from  a health  maintenance  organization, 
and  Internists  and  Amosists  (military  corpsmen  who  receive  brief 
medical  training  and  are  taught  to  use  a number  of  explicit  clinical 
algorithms)  from  a major  military  base.  Concentrating  on  only  cer- 
tain aspects  of  this  extensive  study,  we  note  that,  in  comparison  to 
the  physicians,  Medex  ordered  a variety  of  diagnostic  tests  as 
appropriately  as  physicians,  and  their  use  of  antibiotics  was  some- 
times as  appropriate  and  sometimes  clearly  more  appropriate  than 
that  of  physicians.  (Tompkins  et  al.,  1977) 

As  part  of  an  experiment  utilizing  nurse  practitioners  in 
office  settings,  Sibley  et  al.  (1975)  measured  and  specifically 
compared  the  quality  of  clinical  care  provided  by  physicians  and 
nurse  practitioners.  Appraisal  was  carried  out  by  abstraction  of 
data  from  charts.  Explicit  criteria  for  proper  management  of  selected 
"indicator"  conditions  (e.g. , otitis  media,  hypertension,  prenatal 
care,  depression,  urinary  tract  infection,  knee  Injury)  and  for 
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appropriate  use  of  drugs  (e.g.,  chloramphenicol,  steroids,  tetra- 
cycline, vitamin  B^*  tran<lum*er8»  cardiac  glycosides)  were  then 
applied.  The  investigators  were  able  to  show  that  none  of  the 
observed  physiclan/nurse-practltloner  differences  in  adequacy  of  care 
was  significant  at  the  5 percent  level.  Further  support  for  the 
adequacy  of  nurse  practitioner  care  came  from  an  analysis  "to  assess 
the  likelihood  that  the  adequacy  of  care  could,  in  fact,  be  worse  in 
the  [nurse  practitioner]  practice.  The  analysis  showed  that  the 
probability  of  a true  deterioration  of  10  percent  or  more  in  the 


quality  of  care  in  the  [nurse  practitioner]  practice  compared  with 
the  [control]  practice  was  [only]  0.018  for  indicator  conditions  and 
0.072  for  drugs." 

Evaluation  of  physician's  assistants  by  means  of  direct  observa- 
tion of  technical  process  of  care  was  carried  out  by  Duttera  and 
Harlan  (1978)  at  ten  rural  primary  care  practice  sites.  When  physi- 
cian's assistants  (both  physician  associates  and  Medex)  were  compared 
to  their  physician  employers  for  appropriateness  of  diagnosis  and 
therapy,  Duttera  and  Harlan  found  that  the  two  groups  of  providers 
"handled  less  severe  and  more  severe  encounters  equally  well,  with 
no  statistically  significant  difference  in  performance."  (Also  of 
interest,  though  of  less  pertinence  to  this  review,  is  that  "scores 
of  physician  assistants  were  correlated  with  those  of  their  physician 
employers  (r  ■ .61,  P < .01.)" 

An  innovative  approach  to  the  question  of  technical  competence 
of  nurse  practitioners  was  carried  out  by  Perrin  and  Goodman  (1978) 
in  their  study  of  management  over  the  telephone  of  five  common  acute 
pediatric  problems,  by  pediatricians  in  practice,  pediatric  house 
officers,  and  pediatric  nurse  practitioners.  The  providers  were 
scored  for  technical  and  non-technical  aspects  of  care,  including 
relevance  and  completeness  of  historical  information  gathered,  and 
correctness  of  disposition  suggested,  as  well  as  telephone-interviewing 
skill.  The  pediatric  nurse  practitioners  scored  significantly  higher 
in  all  aspects  of  telephone  management.  They  were  also  no  less  effi- 
cient than  the  pediatricians  and  pediatric  house  officers;  that  is, 
they  "gathered  historical  data  and  suggested  therapy  with  no  more 
unnecessary  questioning  or  discussion  than  physicians." 
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liicrf  li.-iN  boon  one  published  study  (Kane,  Olsen,  Castle,  197ft) 

In  which  the  technical  process  of  care  of  physician  extenders,  speci- 
fically Medex,  was  compared  with  the  technical  process  of  care  pro- 
vided by  their  physician  preceptors,  through  the  use  of  an  encounter 
form  (patient  contact  record).  When  criteria  were  established  for 
the  correct  use  of  laboratory  tests  and  appropriateness  of  treatments 
ordered,  there  was  no  significant  difference  noted  In  six  of  the  ten 
process  measures  used,  between  the  care  rendered  by  Medex  and  their 
supervising  physicians.  In  three  of  the  process  measures,  the  diff- 
erence between  the  physician  and  Medex  was  significant  at  0.05  < P < 

.10,  with  the  trend  favoring  higher  quality  by  Medex;  and  for  one 
process  measure,  the  difference  between  physician  and  Medex  was  sig- 
nificant at  P < .05,  indicating  significantly  higher  quality  provided 

i 

by  Medex  (see  Table  12).  Kane,  Olsen  and  Castle  therefore  concluded 
that  "the  Medex  is  more  likely  to  use  appropriate  laboratory  tests 
and  less  likely  to  use  inappropriate  treatments."  Although  their 
conclusion  might  appear  somewhat  strong,  these  data  do  indicate  that 
Medex  provided  a technical  process  of  care,  in  these  rural  Utah  prac- 
tices, of  at  least  as  high  a quality  as  that  provided  by  their  phy- 
sician preceptors.  Kane,  Olsen  and  Castle's  study  also  demonstrated 
the  feasibility  of  using  encounter  forms  to  collect  data  to  examine 
the  quality  of  care.  As  we  have  done,  they  also  cautioned:  "The 
data  are  presented  only  as  indicators  of  patterns  of  care  and,  as 
such,  should  be  interpreted  cautiously.  Our  data  represent  what  was 
checked  off  by  the  physicians  and  Medex  on  the  preprinted  patient 
encounter  forms;  no  means  to  verify  the  accuracy  of  diagnosis  or 
reporting  was  possible." 

In  summary,  the  quality  of  care  rendered  by  physician  extenders 
has  indeed  attracted  attention  but  relatively  little  study  has  been 
carried  out  concerning  the  technical  process  of  care  rendered  by 
physician  extenders.  The  available  Information  suggests  that,  for 
the  health  problems  they  manage,  in  diagnostic  capability,  laboratory 
ordering,  and  treatment  decisions,  physician  extenders  render  a quality 
of  care  equivalent  to  that  of  their  supervising  physicians. 


Process  Measures  of  Quality  of  Care:  Use  of  Laboratory  Tests 
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APPENDIX  C 


ECG  Group 

Thyroid  disease 

Diabetes  mellitus 

Ischemic  heart  diseases 

Arrhythmias  or  heart  block 

0 her  heart  diseases 

Hypertension 

Syncope 

Cholecystitis 

Heart  murmur 


X-ray  Group 
Asthma 

Tension  headache 
Migraine  headache 
Ischemic  heart  diseases 
Arrhythmias  or  heartblock 
Other  heart  diseases 
Acute  sinusitis 
Pneumonia,  pneumonitis 
Acute  bronchitis 

Chronic  bronchitis,  emphysema,  COPD 

Chronic  sinusitis 

Cholecystitis 

Osteoarthritis 

Rheumatoid  arthritis 

Other  arthritis/rheumatism 

Bursitis,  tenosynovitis,  synovitis 

Backache  with  sciatica 

Backache  alone 

Pain  in  joint 

Other  headache 

Fracture  of  lower  limb 

Fracture  of  upper  limb 

Dislocation,  upper  extremity 

Other  fracture 

Sprain/strain  upper  limb 

Sprain/ strain  lower  limb 

Sprain/strain  neck/back 

Heart  murmur 

Dislocation,  lower  extremity 
Trauma  to  head 
Muscle  pain 

Other  bones.  Joints,  muscles 
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Phyaical  Therapy  Group 

Osteoarthritis 
Rheumatoid  arthritis 
Other  arthritis/rheumatism 
Bursitis,  tenosynovitis,  synovitis 
Backache  with  sciatica 
Backache  alone 
Pain  in  joint 
Fracture  of  lower  limb 
Fracture  of  upper  limb 
Dislocation,  upper  extremity 
Other  fracture 
Sprlin/straln  upper  limb 
Sprain/strain  lower  limb 
Sprain/ strain  neck/back 
Dislocation,  lower  extremity 
Other  bones,  joints,  muscles 
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